Joining formative evaluationwith translational science to assess an EBI in foster care: Examining social–emotional well-being and placement stability

a r t i c l e i n f o Keywords: Formative evaluation Translational research Cultural exchange Foster care Evidence-based interventions Social–emotional well-being Placement stability This study examined measures of placement stability and social–emotional well-being for a federally-funded demonstration project of inhome Parent Management Training-Oregon model (PMTO) for children in foster care with serious emotional disturbance (SED). Following a cultural exchange framework for translational research and a tollgate approach to evaluation, this formative study tested these measures prior to further investment in summative evaluation. The research aim was to observe whether measures of social–emotional well-being and placement stability performed as expected. Using a pretest–posttest randomized consent trial, children identified as SED within six months of entering foster care were randomly assigned to PMTO or to a services-as-usual comparison group (N = 121). A multi-group structural equation model was tested to observe the relationship of baseline social–emotional wellbeing and placement stability on post-test social–emotional well-being. Results showed that post-test well-being was significantly predicted by baseline well-being for both groups and, importantly, only the intervention group demonstrated significant effects of placement stability on post-test well-being. For the intervention group, as placement stability increased, post-test social skills significantly improved, demonstrating an association between well-being and placement stability that was not evident in the comparison group. Overall, wellbeing measures performed as expected and detected relationships between variables and variation as hypothesized. Additionally, placement stability may have been mediated by the PMTO intervention. In conclusion, by using a real world example of the translational research concept of cultural exchange between university-based researchers and agency-based practitioners, this study shows that formative evaluation offers an important opportunity to test ad hoc research questions inspired by the process of implementation itself. An increasing drive for evidence-based interventions (EBIs) has resulted in heavy investments by federal, state, and local stakeholders to develop, implement, and evaluate innovations in child welfare. Too often, however, the process of implementation and evaluation encounters unanticipated setbacks and hurdles that limit the quality and quantity of information garnered from the research. Even EBIs which find strong empirical support in one setting may appear totally ineffective in another, particularly when comparing a quasi-or non-experimental study to a randomized controlled trial (RCT) (Epstein & Klerman, 2013). Moreover, the research-to-practice gap of a nearly 20-year time lag between successful clinical trials of an EBI and its widespread implementation motivates the field to take additional actions to expedite …